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I. INTRODUCTION 


In Brazil there are many navigable 


www.ijaers.com 


waterways, 
however, according to a study prepared by the National 
Transport Confederation (CNT, 2019), only 30% of the 
waterways are used. In numerical question, of the 63 
thousand kilometers of rivers with navigation potential, 
only 19.5 thousand kilometers are actually used. From the 
study, it can also be stated that in the Amazon region, 
about 10 million passengers are transported per year and 


Abstract— There are many rules to be followed to assess the safety of 
navigation, the certifiers and classifiers are responsible for ensuring 
compliance with all these rules that ensure the integrity of the vessels, 
however, this is not enough. The Naval District, in which the state of Para 
is included, was the first in accidents that occurred in the year 2020 and 
the third in the year 2021. Due to these accident occurrences, concepts of 
artificial intelligence, machine learning and deep learning were applied in 
this area. Aiming to assist in this process, this work proposes to develop an 
application using Convolutional Neural Network (CNN) for image 
recognition (Vessels and plimsoll disk). In this sense, a Convolutional 
Neural Network (CNN) learning technique was used to identify the type of 
ship through a bank of supplied images, the same method was applied to 
identify if there is accident risk with the ship through the analysis of 
plimsoll disk images. To perform the training of the CNNs, six different 
network architectures were evaluated with: changing the number of filters 
in each convolutional layer; varying the amount of convolutional layers 
and; using transfer learning of the VGG-16 network with the fine tuning 
technique. The results achieved in this work are promising and 
demonstrate the feasibility of employing Convolutional Neural Network as 
a method for identifying the images of vessels as from the plimsoll disk). 


that in the last 8 years, until the year of the referred study, 
there was a growth of 34.8% in cargo transportation. 


However, with the increase in demand for fluvial 
transport, the probability of accidents increases. The 
Navy, in turn, has prepared a document called the 
Administrative Inquiry on Accidents and Facts of 
Navigation ([AFN), which collects and organizes data on 
shipping accidents according to: nature, type of vessel, 
number of vessels per year, naval district. 


Page | 64 


Oliveira and Oliveira 


The Naval Districts, according to Decree No. 2,153, 
have many attributions, one of them being to monitor 
maritime traffic. In order to cover the whole Brazilian 
territory 9 naval districts were created. The 4th Naval 
District is responsible for the states of Amapá (AP), 
Maranhão (MA), Piauí (PI) and Para (PA), the latter being 
the study region. In 2020 about 34% of the occurrences 
belong to the 4th naval district responsible for part of the 
northern states, being in Ist place among the districts. 
Some of the causes that lead to accidents in rivers are due 
to the fact that boats, this being the means of transportation 
that had more occurrences, due to excess weight or wrong 
positioning of cargo and passengers 


The total weight and the position of the weights 
directly influence the stability of a vessel and to be able to 
be evaluated, there are analysis criteria, for this case, the 
Freeboard, which is the distance from the waterline to the 
main deck and the transverse tilt will be addressed. About 
the positioning of the loads, not being done properly can 
lead to an inclination to one of the sides, which are the 
sides of the ship, called heeling. 


When a vessel tackles it is possible to observe that one 
of the sides ends up with a low freeboard height, 
according to examples from Dokkun (2013b), and thus 
ends up not passing the requirements established by the 
Norms of Maritime Authority (NORMAM). In the case of 
excess passengers, the freeboard already decreases, but it 
gets even worse when the cargo and passengers are 
positioned incorrectly, which is very common, this 
increases the risk of people falling into the river and even 
the ship sinking. For this reason, it is very important to 
maintain a minimum height for the freeboard, as provided 
by the Norm of Maritime Authority for Inland Navigation 
- NORMAM 02 (MARINHA DO BRASIL, 2005). 


To evaluate the risks of a vessel accident due to 
freeboard, one of the Deep Learning techniques will be 
used, which are called Convolutional Neural Networks 
(CNN). This programming technique is inserted within 
the Artificial Intelligence (A.I.) environment, to which 
has shown to be very effective for problems with images, 
such as: classification, pattern recognition 
(RONNEBERGER et al., 2015), character extraction 
(OCR's) (MICROSOFT, 2018) and object detection 
(GOOGLE, 2017), ship recognition using distributed self- 
organizing map (LOBO, 2009). 


Based on the data provided by the Navy and the 
advances in Deep Learning techniques, the goal is to 
develop a CNN code capable of classifying images of 
regional vessels, from an analysis of the free edge and 
finally determining if the condition of the vessel presents 
a risk of loss of stability. Therefore, with the advance of 
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computing and techniques of artificial intelligence, such 
as Neural Networks, and with the analysis of the history 
of occurrence of accidents in navigation obtained by the 
IAFN data, it would be important to raise the question: Is 
it possible that an AI with so many contributions to 
society, collaborate with the reduction of accident rates 
warning passengers about the risk of accidents before 
sailing? 
Il. THEORETICAL REFERENCE 


Image recognition has been widely used in several 
branches of research, such as in medicine, for diagnosing 
adisease from the analysis of test results; in agriculture, 
applied in the identification of pests in crops; in 
autonomous cars, replacing human vision for decision 
making; among other applications (FERREIRA, 2017; 
PRETO, 2018; TRINDADE, 2018; VOGADO et al., 
2019). 


One of the methods that has been successfully applied in 
the processing and analysis of digital images, with high 
effectiveness to solve problems of pattern and object 
recognition in images, is the Convolutional Neural 
Network (CNN) (ALMEIDA, 2018). Aiming at 
identifyingthe components of a railway line, as well as the 
indication of disintegration and chips of its sleepers, from 
image analysis, Gilbert, Patel and Chellappa (2015) 
applied fully Convolutional Neural Networks for 
classification of 10material classes. This architecture was 
proposed for the challenge of extracting accurate 
information from images generated by cameras installed 
in moving vehicles, which present distorted backgrounds 
in the railway environment. The approach of this work 
resulted in a material classification accuracy of 93.35% 
and proved the high capacity of Convolutional Neural 
Networks to capture more complex patterns, while 
reusing learned patterns with increasing levels of 
abstraction that are shared among all classes. Following 
the same line, Faghih-Roohi et al. (2016) applied a 
Convolutional Neural Network (CNN) solution for 
automatic detection of rail surface defects. In this work, 
raw images were used as input to the classification model 
and applied to three different CNN structures (small, 
medium and large) in size and number of parameters, in 
which they were compared for classification accuracy and 
computation time. The network was optimized using a 
minilot gradient descent method. In this paper, three types 
of the experienced results were reported, initially with six 
16 classification classes (normal, weld, light defect, 
medium defect, large defect, and joint) where a high 
percentage of false results were found. Since then, the 
normal and weld classes have been integrated as Normal 
and all three defect classes as a single class. In this way, a 
new performance evaluation of the CNN models was 
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presented with the three classes: Normal, Defect, and 
Joint. The performance of the three CNN models (small, 
medium and large) for the multi-class (3 classes) and 
binary (normal and defect) classifications based on the 
averages of the calculated performance metrics and the 
corresponding standard deviations were also presented. As 
a result, the defect classification performance achieved 
almost 92% accuracy across the three network structures, 
with the large one having the highest value and the longest 
time spent for training. It was found that the two-class 
training showed a difference of approximately 1% higher 
accuracy compared to three- class, which validated a good 
performance in multi-class training. (FAGHIH-ROOHI et 
al., 2016). 


2.1 Artificial Neural Network 


To understand how the Artificial Neural Network works, it 
is necessary to understand a little about the human brain. 
The human brain can identify images, sounds, and patterns 
through stimuli generated by the external environment. 
Behind this action and reaction of the person in relation to 
the environment, there is a sequence of processes that 
occur in neurons. According to Moreira (2017), biological 
neurons, represented by Figure 1, are basically composed 
of dendrites, cell body, and axon: 


Fig 1: Biological Neuron 
Source: Fausset (1994, p.6) 


The dendrites (a) are responsible for receiving stimuli from 
other neurons or sensory cells, adjusting the impulses 
coming from other neurons. The cell body (b) contains the 
nucleus, where protein synthesis occurs. In the axon (c) is 
transmitted the nerve impulses from the cell body, while the 
endings of the axon (d) is where it contains the synapses, 
which establish communication with the dendrites or cell 
bodies of other neurons (Moreira, 2013). From this it was 
developed a system that mimics the neurons biological 
neurons, called artificial neurons, as representedin Figure 2 
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Fig.2: Artificial Neuron 
Source: Authors (2021) 


2.2 Conventional Neural Networks (CNN) 


Among the various deep learning architectures, 
Convolutional Neural Networks (CNNs) have been 
achieving remarkable prominence in their main areas, 
having major advances in audio, video, and image 
processing (LECUN; BENGIO; HINTON, 2015). 
CNNs have been inspired by the hierarchical organization 
of the visual cortex and the classical notions of simple 
and complex cells. According to Hubel and Wiesel's 
(1968) experiments with cats and monkeys, it was 
observed that simple cells were activated when presented 
with simple patterns to the animal, such as lines; and 
complex cells were activated when more elaborate 
patterns, composed of combinations of simple patterns, 
were presented to the animal (HUBEL; WIESEL, 1968; 
LECUN; BENGIO; HINTON, 2015). 


Il. METHODOLOGY 


For the identification of images by the CNN, it is 
necessary to go through a few steps, which consists of 
collecting images, training the neural network and 
evaluating the results and for this will follow a few steps 
until the end of the process. However, before detailing the 
procedure, it is important to present the tools and the 
architecture of the network 


3.1 Tools 


For the development and training of the network, a 
computer with the following configurations was used: 
Intel(R) Core (TM) i5-9300H CPU @ 2.40GHz, 8.00 
Gigabyteof RAM and NVIDIA GTX 1650 card with 
4 Gigas memory, with Windows 10 operating system. 


The Python language plataforma used handling the 
libraries, environments and basic programming settings 
was Anaconda, due to its ease of handling. Within the 
platform an integrated development environment (IDE) 
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was provided together with the facility to install the 
necessary libraries. 


- Python: It is a high-level, dynamic, interpreted, 
modular, multiplatform and object-oriented 
programming language. It has a simple and easy-to- 
understand syntax and a wide variety of libraries 
(PYSCIENCE BRAZIL, 2022). 


- Spyder: The IDE has a very intuitive interface for 
handling and programming. It has an advanced 
combination of editing, analysis, debugging, and 
profiling. Easy data exploration, deep inspection, and 
visualization features (SPYDER, 2022). 


- TensorFlow: An open source platform that facilitates 
the development and implementation of Machine 
Learning (Machine Learning) (TENSORFLOW, 
2022). 


In addition to the TensorFlow/Keras library, the OpenCV, 
Numpy, Matplotlib, Scikit-learn and Imbalanced-learn 
libraries were used. 


- OpenCV: also known as Open Source Computer 
Vision, is an applied image processing library for 
developing applications in the field of Computer 
Vision (OLIVEIRA, 2019); 


- Numpy: multidimensional matrix processing 
package, which provides several tools to manipulate 
and manage them (OLIVEIRA, 2019); 


- Matplotlib: biblioteca de software para criação de 
gráficos bidimensionais de visualização de dados 
(OLIVEIRA, 2019); 


- Scikit-learn: Open source machine learning library 
that supports supervised and unsupervised learning. It 
also provides various tools for model fitting, data 
preprocessing, cross-validation, model selection and 
evaluation, among many other utilities; 


- Imbalanced-learn: open-source machine learning 
library that supports supervised and unsupervised learning. 
It also provides various tools for model fitting, data 
preprocessing, cross-validation, model selection and 
evaluation, among many other utilities found in machine 
learning and pattern recognition applications 


The methods are categorized into, subsampling, where 
samples from the majority class are discarded during the 
balancing procedure; oversampling, where new samples 
are generated in the minority class to achieve a certain 
balancing proportion; and a combination between 
subsampling and oversampling to balance the samples 
(LEMAITRE; NOGUEIRA; ARIDAS, 2017). 


3.2 Network Architecture 


In this work, two network structures were developed, one 
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for vessel classification and the other for the plimsoll disk. 
The network structure is composed of 4 layers, being 3 
convolution neuron layers and 1 fully connected neuron 
layer. The first layer is composed of 32 neurons (filter) 
with dimension 3x3, the second has 64, the third has 128. 
The first fully connected layer has 512 neurons that then 
passes to the second layer also fully connected that has 6 
neurons, which are equivalent to the amount of 
responsefor vessel classification, Table 1. 


Table 1: Network architecture for vessel classification 


Convolution N. of Filters. 32 


Layer 1 Dimension of Filters 3x3 
Max-pooling 


Dimension of Filters 2x2 
Layer 1 


Convolution N. of Filters 64 


Layer 2 Dimension of Filters 3x3 


Max-pooling 


Dimension of Filters. 2x2 
Layer 2 
Convolution. N. of Filters 128 


Layer 3 Dimension of Filters. 3x3 


Max-pooling 
Dimension of Filters 2x2 
Layer 3 


Fully Connected 
N. of nodes 512 
Layer 1 


Fully Connected 
N. of nodes 6 
Layer 2 


Table 2: Network Architecture for Plimsoll disk 


classification 
Convolution N. of Filters 32 


Layer 1 Dimension of Filters 3x3 


Max-pooling 
Dimension of Filters 2x2 
Layer 1 


Convolution N- of Filters 64 


Layer 2 Dimension of Filters. 3x3 
Max-pooling 


Dimension of Filters 2x2 
Layer 2 


Convolution N. of Filters, 128 


Layer 3 Dimension of Filters 3x3 


Max-pooling 
Dimension of Filters 2x2 
Layer 3 


Fully Connected 
N. of nodes 512 
Layer 1 


Fully Connected 


N. of nodes 3 
Layer. 2 


Meanwhile, for the Plimsoll disk classification, the same 
network structure was used, but in the second fully 
connected layer, only 3 neurons were used, Table 2. It is 
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worth noting that the first 3 layers of the network have the 
Max-pooling activation function with 2x2 dimensions. 


3.3 Processing Flow 


The following procedure will be used for the CNN, 
represented by flowchart 1: 


Step’ *Collect images from internet 

` *The images are of vessels according to the classes specified in CNN 

(Collecting 
images), 


*Database with classes to training 
| Step?  eDatabase with classes to validating 
(Database) 


NW *Discretization of CNN 


step 3 *Definição de variáveis 
( CNN) Activation Function 


A Train the CNN 


Step4 — *Analyze the generated precision 
(Train and *Check the need to improve the dataset, increase in more neurons or increase the number of Epochs 
Validate, 


*Perform test with random images to evaluate the behavior of CNN 
Step5 eCheck the necessity to improve the dataset 
(Test) 


Fig 3: Flowchart I — Stage of the method applied to CNN 
Source: Own Authors (2021) 


Fig.4: Example of Imagens used of the vessels — Database 
Source: Own Authors (2021) 
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In step 1, we collected images from various sources, all 
specifically of the classes determined within the 
programming, which are Military Ship, Ferry Boat, Canoe, 
Catamaran, Sailboat and Yacht, as seen in section 3.1 and 
for the Plimsoll Disc, section 3.3. In step 2 we created a 
Database (Database) containing numerous images from 
the Internet, of the ships, Figure 4 and Figure 5, and of the 
plimsoll disc, Figure 6 and 7. 


Fig.5: Example of Imagens used of the vessels — Database 
Source: Own Authors (2021) 


Fig.6: Example of Images used from plimsoll disk - 
Database 


Source: Own Authors (2021) 


All photographs are different for training and validation, 
asit is important that the program can refine its parameters 
based on various perspective points of the ship type, 
without risking biased training. 


After the elaboration of the Dataset of images, following 
to step 3, the development of the code for image 
classification will be prepared in Python language, whose 
platform is considered Open Source (Open Resources), 
allowing users to use it for free, besides being able 
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to enjoy numerous libraries, one of them being the 
TensorFlow (TF), created by Google Brain team. TF is an 
open source library for Machine Learning and the 
Application Programming Interface (API) that will be 
used for Deep Learning ‘and Keras, which has a set of 
programming routines (GOOGLE, 2015). Both libraries 
will be imported by within the Python platform. In this 
step, we will discretize the Convolutional Neural Network 
(CNN) process, including. The number of neurons, layers, 
activation functions, epoch and the way the results are 
displayed on the screen 


After configuring the CNN and proceeding to step 4, 
the tool will be executed, where it will train and validate 
its parameters for classification, trying to discard 
irrelevant features, for example the presence of water, and 
prioritize specific contents, such as hull shape. For details 
about its accuracy, the percentage will be displayed on 
screen during all training epochs. 


Finally, step 5 consists of performing a test in which 
random images are presented, which are not in the 
database, and generating a prediction of the results. In this 
case, if there is any inconsistency in the results, this error 
may lead to the interpretation that it may be related to the 
photographs in the database or the number of epochs 
generated. This method will be applied both for the 
classification of the vessel type and for the analysis of the 
freeboard of the vessel. 


IV. ANALYSIS OF THE RESULTS 


Images of six types of ships were collected, as seen in 
section 3.1, and a database was created for the CNN to 
use for training. For the method mentioned in section 4, in 
step3, the settings regarding classification is specified in a 
categorical way, because there are more than two 
classesto be evaluated. 


As the images obtained have very large dimensions, 
requiring a high demand of time and computational effort 
to process the pixels, they were resized to dimensions of 
200 pixels long and 200 pixels wide, acceptable sizes for 
conventional computers. It was also determined that 5 
layers of neurons would be used, the first containing 32 
neurons, the second with 64 neurons, the third with 128 
neurons, and the fourth with 512 neurons. The activation 
functions for these specified layers contain the ReLU 
function. 


The fifth and last layer is the output layer, so the 
numberof neurons is equal to the number of inputs. In 
this case, six classes were used, which are the types of 
vessels, and the activation function is Softmax. For the 
loss function the Categorical Cross Entropy was used, 
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which collaborated with the evaluation of the epochs, to 
determine if the results were converging. After the 
configuration is done, in step 4 the CNN is trained. 
During the training the accuracy and loss values are 
generated for each (Epoch), as per Figure 7, for both 
training and validation. 


- 2s 1s/step - loss: 0.4745 - 


- val_accuracy: @.4231 


Fig.7: Accuracy and loss values 


Source: Own Authors (2021) 


(b) 


0 50 100 150 (c) 
Fig 8: a) Military Ship, b) Canoe, c) Yacht 
Source: Google (2021) 


If after all epochs the accuracy value is not acceptable, the 
possibility of increasing the quantity of images to improve 
the distinction will be verified, or more layers of neurons 
will be added, or the number of epochs will be 
increased. If after training, everything is as expected, in 
step 5 the CNN will be provided with random images of 
vessels that are not in the database so that it can classify. 


The Convolutional Neural Network (CNN) was 
programmed to classify images as to the type of ship. In 
the figures below results were obtained for photographs of 
vessels in which the correct order of responses would be: 
Military Ship (Figure 8-a), Canoe (Figure 8-b), Yacht 
(Figure 8-c), Sailboat (Figure 9-a), Canoe (Figure 9-b), 
andCatamaran (Figure 9-c). Tests were conducted with 30, 
50 and 75 epochs and all with 2 steps per epoch, Figures 
10, 11 and 12 respectively. 
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Fig.9: a) Saillboat, b) Canoe, c) Catamaran 
Source: Google (2021) 


Fig.11: Prediction result for 50 epoch training 
Source: Own Authors (2021) 


Fig.10: Prediction result for 30 epoch training 
Source: Own Authors (2021) 


Fig.12: Prediction result for 7 epoch training 
Source: Own Authors (2021) 
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It can be observed that there was an error in the last 
classification for training periods of 30 and 50 epochs. 
However, for 75 epochs the network was correct in all its 
classifications. 


For the Plimsoll Disk 3 types of loading were considered: 
light, loaded and overloaded, as specified in section 2.3. 
The number of neurons remained the same, but the number 
of epochs became 20 and the steps per epoch 5. Two cases 
were considered, the first in which 6 random images of the 
Plimsoll disk were provided for the neural network to 


analyze, see Figures 13 and 14. 


(b) 


© 
Fig.13: a) Lignt; b) Light; c) Light 
Source: Google (2021) 


Figures 12 and 13 show the order of images that was 
provided to the neural network, which the right answer 'e: 
Light, Light, Light, Light, Overloaded, and Overloaded. 
After the images were provided for the network to analyze, 
the following results were generated, Figure 14. 


The neural network responded correctly in relation to the 
provided images, which can be observed that it achieved 
satisfactory results with respect to the predictions in 
different types of images and conditions. Next, taking 
advantage of the network, a case of a specific vessel was 
simulated, two images representing the In this case, it is 
possible to simulate a real situation, in thecase of having 
images of the edges of the same ship, but one with the 
real image and the other with the same image, but 
modified so that it could simulate a real situation, in this 
case having images of the edges of the same ship. In this 
ship the one edge is overloaded, Figure 15a, and the other 
edge is in a light condition, Figure 15b, which represents 
a ship in web condition. 
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Fig.14: Prediction result for training 20 epochs with 5 
steps per epoch 


Source: Own Authors (2021) 


(b) 
Fig.15: a) Overloaded; b) Light 
Source: Google (2021) 


After submitting the images to the neural network, it 
generated the following results, Figure 16. It can be 
observed that in a real condition, where it is provided 
images that belong to the edges of the same vessel, the 
neural network was correct in its predictions, which were: 
Overloaded and Light. 


Fig.16: Simulation of the plimsoll disk from the same 


vessel 


Source: Own Authors (2021) 
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V. CONCLUSION 


With the development of this tool it was possible to assign a 
methodology using one of the areas Artificial Intelligence 
that is Deep Learning. Through this it was possible to create 
a convolutional neural network that could classify as to the 
type of vessel and the loading of the vessel through the 
plimsoll disk. Artificial intelligence has been increasingly 
used in all market sectors. In this case, it can be improved 
more and more to be used in ensuring the safety of 
navigation, which can impact in the future on a reduction of 
accidents. Not only through image analysis, but it can also 
be expanded and used in other ways. It can also be 
observed that the Convolutional Neural Network presented 
satisfactory results, despite the database not being vast, 
which normally on average reaches thousands of images, 
but it was able to train the network and learn to classify the 
types of vessels in different situations and the plimsoll disk 
in 3 types ofloading: Light, Loaded and Overloaded. It was 
collected images from the Internet and from the region for 
the elaboration of the database, despite the difficulty in 
arranging data from the plimsoll disk, due to the access to 
the vessels and the overloaded condition that is not 
common to be observed, it can be circumvented through 
the use of synthetic data. 


For future works, it is possible to develop new networks or 
improve the CNN of this work to be analyzed in real time 
images, structure analysis, search for ship documents, 
increase the database to classify more types of ships, use 
neural networks in port security, observing ships that dock. 
The use of neural networks is quite vastapesar da 
dificuldade em arranjar dados do disco de plimsoll, devido 
ao acesso as embarcações e a condição de sobrecarga que 
não é comum de ser observado, pode-se contornar através 
do uso de dados sintéticos. 
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